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The  Need  for  Change 

ORGANIZATION 

Our  methods  and  tools  must  respond  to  a 

changing  world: 

•  Large  number  of  scenarios  required  due  to  an 
uncertain  future 

•  Increasing  emphasis  on  command  and  control 

•  Growth  of  Operations  Other  Than  War  (OOTW) 
importance  requiring  new  approaches 

•  Sense  of  urgency  leading  to  need  for  easy  to 
use  fast  turnaround  tools 

•  Need  to  integrate  tools  for  training,  acquisition, 
and  operational  support. 
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Categories  of  Methods  and  Tools 


•  Data  Collection/Generation  -  Used  to  collect  or 
generate  data  for  subsequent  analysis. 


•  Data  Organization/Relationship  -  Used  to 
organize  data  or  to  establish  relationships 
between  data. 


•  “Solving”  -  Used  to  mathematically  derive 
solutions  to  problems,  even  if  based  on 
subjective  data. 

*  Support  -  Used  to  organize,  store,  and  explore 
typically  large  sets  of  empirical  data. 


Any  analytic  method/tool  has  potential 
application  for  C2  assessments 
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Example  Methods/Tools  by 

Category 


Category 

Example  Methods/Tools 

Data 

Collection/Generation 

Real  Word  Operations  Lessons  Learned,  After  Action 
Reviews/Reports,  Historical  Analysis,  Expert  Elicitation, 
Constructive  (Force-On-Force) ,  Virtual  (Simulators, 
STAFFEX,  MAPEX),  and  Live  (CPX,  FTX,  Advanced 
Warfighting  Experiments)  Simulations,  Game  Theory, 
Wargaming,  Brainstorming,  ACCESS/HEAT 

Data  Organization 
and  Relationships 

Causal  Mapping,  Multi-Criteria  Decision  Analysis, 
Regression  Analysis,  Factor  Analysis,  Neural  Nets, 
Systemic  Approaches  (e.g.  Systems  Dynamics) 

“Solving” 

Mathematical  Analysis,  Linear  Programming,  Goal 
Programming,  Integer  Programming,  Heuristic 

Searching  Techniques,  Genetic  Algorithms,  Project 
Management  Tools 

Support 

Data  Analysis,  Geographical  Information  Systems, 
Visualization,  Databases,  Checklists,  Spreadsheets, 
Planning  Tools 
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Types  of  Simulation 

ORGANIZATION 

Constructive  Simulation  (Simulated  people  operating 
simulated  systems): 

Force-on-Force  Simulations  (Analytic  &  Training), 
Specialized  Models,  Network  Simulations  (Petri-Net, 
Live  Analyst,  OPNET) 

Vjrtual  Simulation  (Real  people  operating  simulated 
systems): 

MAPEX,  CAMEX,  STAFFEX,  Simulators,  DEXES/CAM 
Ljve  Simulation  (Real  people  operating  real  systems): 
CPX,  FTX,  AWE 

Source(s)  of  definitions:  -  US  DoD  M&S  Master  Plan ,  October  1995 

-NATO  M&S  Master  Plan,  August  1998 
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Issues  with  Assessing  C2 
Effectiveness  (1  of  2) 

ORGANIZATION  '  ' 

Issues  for  all  C2  analysis,  to  include  OOTW  C2: 

*  Human  behavior  representation 

*  Homogeneous  models  versus  hierarchies/federations 

*  Stochastic  versus  Deterministic  models 

*  Adversarial  representation 

*  Verification,  Validation,  and  Accreditation  of  models 

*  Sensitivity  analysis  and  uncertainty 

These  issues  also  addressed  by  SAS-01 7  LTSS. 
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Issues  with  Assessing  C2 
Effectiveness  (2  of  2) 

ORGANIZATION  '  7 

Issues  particularly  important  for  OOTW  C2  analysis: 

•  Selecting  an  orchestrated  set  of  tools 

•  Scoping  the  analysis 

*  Consideration  of  the  human  dimension  early 

*  Trust/confidence  in  tools  by  decision  makers 
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New  Methods  and  Emerging 

Practices 


ORGANIZATION 


Evaluate  and  select  an  orchestrated  set  of 

methods  and  tools 

*  Apply  models  effectively  in  a  complementary  way 
during  all  phases  of  the  analysis  where  required 

*  Link  methods  and  tools  from  problem  formulation 
through  problem  solution 

*  Federate  models  in  real  time 

*  Link  simple  tools  for  scanning  the  scenario  space  and 
complex  models  for  examination  of  specific  points 

*  Link  a  hierarchy  of  models  when  required 


PR7(A)-9 


Method/Tool  Selection 


Method/Tool  Selection  Criteria 

Functionality-Related  Criteria 

Performance-Related  Criteria 

*  Resolution 

*  Completeness/Scope 

*  Functions  Represented 

*  Explicitness 

*  MoM  Generation  Ability 

*  W&A 

*  Responsiveness 

*  Simplicity 

*  Preparation/Use  Time 

*  Data  Availability 

*  Interoperability 

*  Resource  Requirements 

*  Credibility 
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Using  Models  Effectively  (1  of  2) 


•  Orchestrate  a  set  of  models/tools: 

•  Federate  models  when  possible/appropriate 

*  Establish  a  hierarchy  of  models  (e.g.  link 
performance  models/tools  to  effectiveness 
models/tools) 

•  Select/Develop  models  with  C2  representation: 

•  Represent  C2  processes  through  agent  modeling 
(either  through  rule  sets  or  complexity-based 
approaches) 

*  Represent  the  decision  making  process 
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Using  Models  Effectively  (2  of  2) 


•  Verify,  validate,  and  accredit  models 

•  Verification:  Does  the  model  represent  what  the 
developer  specified? 

•  Validation:  Does  the  model  accurately  represent  the 
real  world  sufficiently  for  intended  use? 

•  Accreditation:  Is  the  model  appropriate  for  its  intended 
analytic  use? 

*  Generate  the  required  data  for  models 


•  Apply  the  models  effectively: 

•  Scan  the  scenario  space  with  appropriate  models 

•  Use  models  in  a  complementary  way  with  other 
models/tools 
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C2  Modeling  Guidelines 


ORGANIZATION 
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Strengths  in  C2  Methods/Tools 

ORGANIZATION 

•  Common  understanding  of  the  issues  and  approaches 

•  Wide  application  of  C2  modeling 

•  Linking  and  federating  models 

•  “Soft  Factor”  analysis  progress 

•  Standards  development 

•  Use  of  Commercial  Off  The  Shelf  (COTS)  products 

•  Application  of  new  information  technologies 

•  Wide  use  of  evolutionary  approaches 
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Remaining/Continuing 
Challenges  (1  of 

ORGANIZATION  ' 

With  C2  method/tgoj  application: 

*  Orchestrating  a  set  of  applicable  tools 

*  Breadth  of  tool  application 

*  Relationship  of  tools  to  data  availability  and  MoM 
generation 

*  Sharing  of  tools  among  communities 

*  Management  of  customer  expectations  and  relations 

*  Consideration  of  M-E-M  or  M-T-M. 
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Model  -  Exercise  -  Model  (M-E-M) 

Approach 


Model 

Exercise 

Model 

S  Assist  train-up 
s  Design  live  scenario 
s  Screen  issues 


s  Focus  data  collection 
s  Enhance  live  play  with 
simulation/stimulation 


S  Extend  experiment 
beyond  live  conditions 
s  Pinpoint  causality  with 
higher  confidence 
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Remaining/Continuing 
Challenges  (2  of  3) 

With  C2  modeling: 

•  Interaction  between  models,  data,  and  measures  of  merit 

•  Better  representation  of  cognitive  processes 

•  Stochastic  versus  deterministic  modeling 

•  C2  terminology  standardization 

•  Scoping  of  C2  analysis 

•  Shared  use  by  analysis,  training,  and  operational 
communities 
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Remaining/Continuing 
Challenges  (3  of  3) 

With  C2  modeling  ( continued ): 

*  Standardization  for  shared  use  by  analysis,  training,  and 
operational  communities 

*  Resourcing  both  the  breadth  and  depth  of  C2  modeling 

*  Similar  representation  of  friendly  and  adversary  forces 

*  Additional  “soft  factor”  modeling 

*  Model  Verification,  Validation,  and  Accreditation 

*  Sensitivity  analysis 
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